Method and apparatus for using rotational movement amount of mobile device and computer-readable recording medium for storing computer program

ABSTRACT

Provided is a method of using a rotational movement amount of a mobile device including obtaining images at two different points on a path along which the mobile device moves, searching for matching points with respect to the obtained images and obtaining an image coordinate value of each of the matching points, sensing linear movement of the mobile device and obtaining a linear movement amount using a result of sensing, and obtaining the rotational movement amount using the image coordinate values and the linear movement amount.

BACKGROUND OF THE INVENTION

This application claims the priority of Korean Patent Application No.2003-95522, filed on Dec. 23, 2003, in the Korean Intellectual PropertyOffice, the disclosure of which is incorporated herein in its entiretyby reference.

1. Field of the Invention

The present invention relates to a mobile device, and more particularly,to a method and apparatus for obtaining an amount of rotational movementof a mobile device and using the same, and a computer-readable recordingmedium for storing a computer program.

2. Description of the Related Art

In general, in order to automatically control a moving object(hereinafter referred to as mobile device), the posture of the mobiledevice should be obtained in real time. Posture means information on theposition and pose of the mobile device. A conventional method in whichthe mobile device itself obtains the posture is largely classified intoone of: a local tracking method and a global localization method. Thesemethods are disclosed for instance on page 10 of a technical reportentitled “Where am I? Sensors and Methods for Mobile Robot Positioning”written by J. Borenstein, H. R. Everett, and L. Feng, published on April1996 and prepared by the University of Michigan for the Oak RidgeNational Laboratories D & D Program.

According to the local tracking method of the disclosed methods, theposture can be simply recognized using an odometry. However, this methodhas a problem of being sensitive to a kinematic modeling error and aunstructured error. To solve the problem, a variety of types of errormodeling and compensation methods have been developed and are disclosedfor example on pages 132-150 of the above technical report, a thesisentitled “Measurement and Correction of Systematic Odometry Errors inMobile Robots” (J. Borenstein, and L. Feng, IEEE Transactions onRobotics and Automation, Vol. 12, No. 6, December 1996, pp. 869-880),and a thesis entitled “Relative Localization using Path OdometryInformation” (Nakju Lett Doh, Howie Choset, and Wan Kyun Chung,submitted to IEEE Transactions on Robotics and Automation). These errormodeling and compensation methods are limited to being applied only whenthe structured errors can be mathematically expressed.

Thus, the position of the mobile device is estimated using a linearaccelerometer while the pose of the mobile device is estimated using agyro or compass. In this case, conventional filter design methods tostably estimate the posture from a sensed signal exist. However, theconventional method of estimating the posture by integrating the sensedresult has problems of not only considerably degrading accuracy inrecognizing posture as time passes due to accumulated integral errors,and but also using expensive sensors.

To solve the above problems, there is a conventional method ofestimating a relative movement of a mobile device using flow of a cameraimage every hour by utilizing a vision system. However, although thismethod is processed by a simple system structure, a calculation time toprocess the image flow is prolonged.

SUMMARY OF THE INVENTION

To solve the above and/or other problems, the present invention providesa method of using an amount of rotational movement of a mobile device byestimating a rotational movement amount from a linear movement amountobtained from the result of sensing linear movement of the mobile deviceand using the estimated rotational movement amount.

The present invention provides an apparatus for using an amount ofrotational movement of a mobile device by estimating a rotationalmovement amount from a linear movement amount obtained from the resultof sensing linear movement of the mobile device and using the estimatedrotational movement amount.

The present invention provides a computer-readable recording medium forstoring a computer program which controls an apparatus for using anamount of rotational movement of a mobile device.

According to an aspect of the present invention, a method of using arotational movement amount of a mobile device comprising obtainingimages at two different points on a path along which the mobile devicemoves, searching for matching points with respect to the obtained imagesand obtaining an image coordinate value of each of the matching points,sensing linear movement of the mobile device and obtaining a linearmovement amount using a result of sensing, and obtaining the rotationalmovement amount using the image coordinate values and the linearmovement amount.

According to another aspect of the present invention, an apparatus forusing a rotational movement amount of a mobile device comprising animage obtaining portion which obtains images from two different pointson a path along which the mobile device moves, a coordinate valuegenerating portion which searches for matching points with respect tothe obtained images and generates an image coordinate value for each ofthe matching points, a sensing portion which. senses linear movement ofthe mobile device, and a movement amount generating portion whichgenerates a linear movement amount from a result of sensing andgenerates the rotational movement amount from the image coordinatevalues and the linear movement amount.

According to yet another aspect of the present invention, acomputer-readable recording medium for storing at least one computerprogram to control an apparatus for using a rotational movement amountof a mobile device, the computer program comprises obtaining images attwo different points on a path along which the mobile device moves,searching for matching points with respect to the obtained images andgenerating an image coordinate value of each of the matching points,sensing linear movement of the mobile device and generating a linearmovement amount using a result of sensing, and generating the rotationalmovement amount using the image coordinate values and the linearmovement amount.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventionwill become more apparent by describing in detail preferred embodimentsthereof with reference to the attached drawings in which:

FIG. 1 is a flow chart for explaining a method of using a rotationalmovement amount of a mobile device according to an embodiment of thepresent invention;

FIG. 2 is a view for explaining the principle of Epipolar Geometry;

FIG. 3 is a flow chart for explaining a method of using a rotationalmovement amount of a mobile device according to another embodiment ofthe present invention;

FIG. 4 is a block diagram illustrating an apparatus for using arotational movement amount of a mobile device according to an embodimentof the present invention; and

FIG. 5 is a block diagram illustrating an apparatus for using arotational movement amount of a mobile device according to anotherembodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Referring to the flow chart of FIG. 1, a method of using a rotationalmovement amount of a mobile device according to an embodiment of thepresent invention includes the steps of initializing a mobile device(Steps 10 and 12), obtaining amounts of a linear movement and arotational movement of the mobile device (Steps 14, 16, 18, and 20),updating posture (Step 22), and determining whether the mobile devicearrives at a target position (Step 24).

In detail, a mobile device starts to move (Step 10). The mobile devicemay be devices moving in 3-D or 2-D, for example, a mobile robot, anArtificial Intelligent automobile, or a unmanned air vehicle.

After Step 10, posture of the mobile device is initialized (Step 12).The posture of the mobile device means information on the position andpose of the mobile device. The initialization of the posture signifiessetting the information on the position and pose of the mobile deviceto, for example, “0” and a movement start point to the origin. AfterStep 12, two images are obtained from the first and second points whichare different from each other on a path along which the mobile devicemoves (Step 14).

After Step 14, matching points between the obtained two images aresearched for and an image coordinate value is obtained and stored withrespect to each of the matching points (Step 16). The matching pointsmay be the minimum required matching points. The number of the minimumrequired matching points may be 8 or 2. Also, according to the presentinvention, to extract an accurate image coordinate in spite of an errorgenerated during the extraction of the image coordinate, an error due toimage noise, and an error due to parameters of a camera (not shown)which obtains an image, the number of the matching points can be set asthe number of the minimum required matching points, that is, an integralmultiple of 8 or 2.

After Step 16, a linear movement of the mobile device is sensed and theamount of the linear movement is obtained using the result of sensing(Step 18). The linear movement amount can be obtained in various formsaccording to the mechanical characteristic of the mobile device, forexample, whether a wheel mechanism of the mobile device moving usingwheels is adopted or whether the mobile device is an air vehicle. Thevarious forms of obtaining the linear movement amount can be realized byusing the techniques disclosed in a book entitled “Sensors for MobileRobots—Theory and Application” (H. R. Everette, A K Peters, Ltd., 1995pp. 35-65) for example, herein incorporated by reference.

According to an aspect of the present invention, the linear movementamount can be obtained directly from the result of sensing of the linearmovement of the mobile device. For example, the acceleration of themobile.device is sensed using an acceleration sensor and the sensedacceleration is integrated twice to obtain the linear movement amount ofthe mobile device.

According to another aspect of the present invention, the linearmovement amount can be obtained indirectly from the result of sensing ofthe linear movement of the mobile device. For example, the linearmovement velocity at the center of the mobile device is obtained fromthe rotational velocity of a wheel of the mobile device and the linearmovement velocity is integrated to calculate the linear movement amountof the mobile device.

After Step 18, a rotational movement amount is obtained using the storedimage coordinate values and the linear movement amount (Step 20).According to the present invention, the rotational movement amount ofthe mobile device can be obtained using a conversion matrix (R_(1,2)) bythe rotational movement amount obtained from Equation 1.The conversionmatrix refers to a matrix to convert a vector component expressed asCoord. 1 to a vector component expressed as Coord. 2.

$\begin{matrix}{{\left\lbrack {\frac{\overset{\_}{u_{i,1}}}{f_{sx}}a_{i}\frac{\overset{\_}{v_{i,1}}}{f_{sy}}} \right\rbrack \cdot R_{2,1} \cdot \left\lbrack {\frac{u_{i,2}}{f_{sx}}1\frac{v_{i,2}}{f_{sy}}} \right\rbrack^{T}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

Here, i=1, . . . , I, I denotes the number of the minimum requiredmatching points, u_(i,2) and v_(i,2) denote the image coordinate valuesobtained in Step 16. R₂, 1 is an inverse matrix of R_(1,2) and denotes anormalized matrix.

$f_{sx} = {{\frac{f_{1}}{s_{x}}\mspace{14mu}{and}\mspace{14mu} f_{sy}} = {\frac{f_{2}}{s_{y}}.}}$f₁ and f₂ denote focal distances from the two points where the twoimages are obtained. s_(x) and s_(y) denote scale factors. T denotes atransposed matrix and a₁ denotes a constant. The normalized matrixrefers to a matrix in which a scale is adjusted such that the size ofeach column vector of the matrix is 1.The

$\left\lbrack {\frac{\overset{\_}{u_{i,1}}}{f_{sx}}a_{i}\frac{\overset{\_}{v_{i,1}}}{f_{sy}}} \right\rbrack$of Equation 1 can be expressed as shown in Equation 2.

$\begin{matrix}{\left\lbrack {\frac{\overset{\_}{u_{i,1}}}{f_{sx}}a_{i}\frac{\overset{\_}{v_{i,1}}}{f_{sy}}} \right\rbrack = {\left\lbrack {\frac{u_{i,1}}{f_{sx}}1\frac{v_{i,1}}{f_{sy}}} \right\rbrack{s\left( {{}_{}^{}{}_{1,2}^{}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

Here, t_(1,2) denotes a linear movement amount and ¹t_(1,2) denotes alinear movement amount expressed as Coord. 1. u_(i,1) and v_(i,1) denotethe image coordinate values obtained in Step 16. s( ) denotes askew-symmetric matrix defined to express the cross product calculationof a vector. The process of obtaining s(¹t_(1,2)) from ¹t_(1,2) isdisclosed in a book entitled “Image Processing, Analysis and MachineVision”p9 (Sonka, V. Hlavac and R. Boyle, PWS Publishing, 1999, p. 462),herein incorporated by reference.

The process of extracting a rotational movement amount from thenormalized matrix (R_(2,1)) is disclosed in a book entitled“Introduction to Robotics” (John J. Craig, Addison-Wesley PublishingCompany, 1986, p. 50), herein incorporated by reference.

The process of obtaining Equation 1 is described below with reference tothe accompanying drawings.

FIG. 2 is a view for explaining the principle of Epipolar Geometry.Referring to FIG. 2, Equation 1 is based on the Epipolar Geometrydisclosed in a book entitled “Image Processing, Analysis and MachineVision” (Sonka, V. Hlavac and R. Boyle, PWS Publishing, 1999, pp.448-465), incorporated herein by reference. A focal point of a camera(not shown) photographing an object 28 in a space is disposed at a point38 and an optical axis is set along a direction 30. When the object 28is photographed in this state, the object 28 is projected on an imageplane 34. An image coordinate (u₁, v₁) of an object 42 projected on theimage plane 34 and a coordinate [¹P₁=(X₁, Y₁, Z₁)] of the object 28 inthe space have a relationship as expressed in Equation 3. Here,^(i)P_(j), wherein i=1 or i=2, denotes a coordinate of a point P_(j) inview of an i coordinate system.

$\begin{matrix}\begin{matrix}{u_{1} = {\frac{f_{sx}}{Y_{1}} \cdot X_{1}}} \\{v_{1} = {\frac{f_{sy}}{Y_{1}} \cdot Z_{1}}}\end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack\end{matrix}$

Here, the unit of each of u₁ and v₁ is the number of pixels.

Likewise, the focal point of a camera photographing the object 28 in aspace is disposed at another point 40 and an optical axis is set along adirection 32. When the object 28 is photographed in this state, theobject 28 is projected on another image plane 36. An image coordinate(u₂, v₂) of an object 44 projected on the image plane 36 and acoordinate [²P₁=(X₂, Y₂, Z₂)] of the object 28 in the space have arelationship as expressed in Equation 4.

$\begin{matrix}\begin{matrix}{u_{2} = {\frac{f_{sx}}{Y_{2}} \cdot X_{2}}} \\{v_{2} = {\frac{f_{sy}}{Y_{2}} \cdot Z_{2}}}\end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

Here, the unit of each of u₂ and v₂ is the number of pixels.

A conversion relationship between the coordinate systems (¹P₁ and ²P₁)with respect to the same object is expressed in Equation 5.¹ P ₁ =R _(2,1)·² P ₁+¹ t _(1,2)  [Equation 5]

Equation 5 can be expressed into Equation 6.

$\begin{matrix}{\left\lbrack {{\frac{Y_{1}}{f_{sx}} \cdot u_{1}}\mspace{14mu} Y_{1}\mspace{14mu}{\frac{Y_{1}}{f_{sy}} \cdot v_{1}}} \right\rbrack^{T} = {{R_{2,1} \cdot \left\lbrack {{\frac{Y_{2}}{f_{sx}} \cdot u_{2}}\mspace{14mu} Y_{2}\mspace{14mu}{\frac{Y_{2}}{f_{sy}} \cdot v_{2}}} \right\rbrack^{T}} + t_{1,2}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

In general, a matter of ego-motion estimation using the EpipolarGeometry refers to estimation of the linear movement amount (t_(1,2))and the matrix (R_(2,1)) using the image coordinate values. Thus,constraints need to be used to remove a term for a coordinate value ofthe point P_(j) in the space from Equation 6.The constraints are thatthe object P_(j) in the space in FIG. 2, Epipoles, and the projectedobjects 42 and 44 all must be present on an Epipolar plane 46corresponding to a shaded portion in FIG. 2. This can be expressed byEquation 7.

$\begin{matrix}{{{{}_{}^{}{}_{}^{}}{\bullet\left\lbrack {{{}_{}^{}{}_{1,2}^{}} \times \left( {R_{2,1} \cdot {{}_{}^{}{}_{}^{}}} \right)} \right\rbrack}} = {0\mspace{34mu}\left( {\bullet\text{:}\mspace{14mu}{inner}\mspace{14mu}{product}} \right)}} & \left\lbrack {{Equation}\mspace{14mu} 7} \right\rbrack \\{{{{}_{}^{}{}_{}^{}} \cdot \left\lbrack {{s\left( {{}_{}^{}{}_{1,2}^{}} \right)} \times \left( {R_{2,1} \cdot {{}_{}^{}{}_{}^{}}} \right)} \right\rbrack} = 0} & \; \\{{Y_{1}{{Y_{2}\left\lbrack {\frac{u_{1}}{f_{sx}}\mspace{14mu} 1\mspace{14mu}\frac{v_{1}}{f_{sy}}} \right\rbrack} \cdot {s\left( {{}_{}^{}{}_{1,2}^{}} \right)}}{R_{2,1} \cdot \left\lbrack {\frac{u_{2}}{f_{sx}}\mspace{14mu} 1\mspace{14mu}\frac{v_{2}}{f_{sy}}} \right\rbrack^{T}}} = 0} & \; \\{{\left\lbrack {\frac{u_{1}}{f_{sx}}\mspace{14mu} 1\mspace{14mu}\frac{v_{1}}{f_{sy}}} \right\rbrack \cdot Q \cdot \left\lbrack {\frac{u_{2}}{f_{sx}}\mspace{14mu} 1\mspace{14mu}\frac{v_{2}}{f_{sy}}} \right\rbrack^{T}} = 0} & \;\end{matrix}$

Q is a final matrix to be obtained. Since the matrix Q has a size of3×3, the number of equations to determine a normalized matrix satisfyingEquation 7 is 8. This means that 8 or more matching points must bepresent in the space to calculate the matrix Q in this embodiment. Therespective components of the matrix Q are obtained from equations todetermine a normalized matrix satisfying Equation 7. This is disclosedin the book “Image Processing, Analysis and Machine Vision” (Sonka, V.Hlavac and R. Boyle, PWS Publishing, 1999, pp. 466-476), for example andherein incorporated by reference.

An algorithm of searching for matching points on two image planes withrespect to a point in a space is disclosed in a book entitled “A RobustTechnique for Matching Two Uncalibrated Images through the Recovery ofthe Unknown Epipolar Geometry” (Z. Zhang, R. Deriche, O. Faugeras, Q.-T.Luong, Artificial Intelligence Journal Vol. 78, 1995, pp. 87-110), forexample, herein incorporated by reference.

Thus, when the linear movement amount of the mobile device obtained fromStep 18 is substituted in Equation 7, the result thereof is expressed inEquation 8. A component of a vector (¹t_(1,2)) can be calculated inadvance through the linear movement amount (t_(1,2)) obtained from Step18.

$\begin{matrix}{{\left\lbrack {\frac{\overset{\_}{u_{1,1}}}{f_{sx}}a_{i}\frac{\overset{\_}{v_{1,1}}}{f_{sy}}} \right\rbrack \cdot R_{2,1} \cdot \left\lbrack {\frac{u_{1,2}}{f_{sx}}1\frac{v_{1,2}}{f_{sy}}} \right\rbrack^{T}} = 0} & \left\lbrack {{Equation}\mspace{14mu} 8} \right\rbrack\end{matrix}$

Here, since the matrix (R_(2,1)) determined by three components (Roll,Pitch, and Yaw) of the rotational movement amount of the mobile devicein the space has a size of 3×3, the matrix is obtained form 8 minimumrequired matching points. Equation 8 is a case in which i=1, which canbe generally expressed in Equation 1.

When the mobile device is limited to move in a 2-D plane, the rotationalmovement amount of the mobile device is determined only by considering arotational component (Yaw) in a vertical direction with respect to themovement plane. Thus, in this case, the number of the minimum requiredmatching points to obtain the normalized matrix (R_(2,1)) can be reducedto 2.

After Step 20, the posture of the mobile device is updated using thelinear movement amount and the rotational movement amount (Step 22).That is, the posture of the mobile device obtained previously, that is,information on the position Trs′ and pose Rot′ of the mobile device, isupdated to a new posture of the mobile device, that is, information onthe position Trs and pose Rot of the mobile device, as shown in Equation9.Trs=Trs′+ΔTrsRot=Rot′+ΔRot  [Equation 9]

Here, ΔTrs denotes a linear movement amount (t_(1,2)) obtained from Step18 and ΔRot denotes the rotational movement amount obtained form Step20.

After Step 22, whether the mobile device arrives at a target position ischecked (Step 24). If the mobile device does not arrive at the targetposition, the process goes back to Step 14. However, if the mobiledevice arrives at the target position, the method of using a rotationalmovement amount of a mobile device shown in FIG. 1 is terminated.

According to the present invention, the method of using a rotationalmovement amount of a mobile device shown in FIG. 1 may selectivelyinclude Steps 10, 12, 22, and 24. That is, the method shown in FIG. 1may include merely Steps 14, 16, 18, and 20, but not Steps 10, 12, 22,and 24. Alternatively, the method shown in FIG. 1 may include merelySteps 14, 16, 18, 20, and 22, but not Steps 10, 12, and 24.

In the method shown in FIG. 1, a finally updated posture can be obtainedby repeatedly executing Step 14 through 22 until the mobile devicearrives the target position. The posture of the mobile device can berecognized through the finally updated posture.

When the time to execute Step 16 of FIG. 1 is prolonged, the mobiledevice needs to stop or slowly move. Thus, there is a need to correct anerror of the posture of the mobile device. A method of using arotational movement amount of a mobile device according to anotherembodiment of the present invention to satisfy the above need isdescribed with reference to the accompanying drawings.

FIG. 3 is a flow chart for explaining a method of using a rotationalmovement amount of a mobile device according to another embodiment ofthe present invention. The method includes the steps of initializing amobile device (Steps 50 and 52), correcting (Steps 54, 56, 58, 60, 62,and 64), that is, updating, posture when the posture needs to becorrected, moving the mobile device when the posture does not need to becorrected (Step 68), and determining whether the mobile device arrivesat a target position (Step 66).

Since the functions of Steps 50 and 52 of FIG. 3 are the same as thoseof Steps 10 and 12 of FIG. 1 and the functions of Steps 56-66 of FIG. 3are the same as those of Step 14-24 of FIG. 1, detailed descriptions onthese steps are omitted. Accordingly, the method shown in FIG. 3 furtherincludes Steps 54 and 68 compared to the method shown in FIG. 1.

In the present embodiment, after Step 52 or if it is determined that themobile device does not arrive at a target position, whether the postureof the mobile device is to be corrected is determined (Step 54). Forthis, for example, whether a time for which the mobile device moveslapses a predetermined time can be determined. The predetermined timecan be set by a user as a time period to correct posture.

If it is determined that the posture of the mobile device is to becorrected, the process goes to Step 56. If it is determined that theposture of the mobile device does not need to be corrected, the mobiledevice is moved and the process goes to Step 66 (Step 68).

While in the method shown in FIG. 1 the mobile device may move duringSteps 16-22, preferably in the method shown in FIG. 3 the mobile deviceshould remain stopped during Steps 58-64. This is because it ispreferred that the mobile device is moved by reflecting accuratelycorrected posture. In addition, in the method shown in FIG. 1, a pathformed two points may be a curve or straight line. However, in themethod shown in FIG. 3, the path formed by two points may be a straightline.

The structure and operation of an apparatus for using a rotationalmovement amount of a mobile device according to an embodiment of thepresent invention are described below with reference to the accompanyingdrawings.

FIG. 4 is a block diagram illustrating an apparatus for using arotational movement amount of a mobile device according to an embodimentof the present invention. Referring to FIG. 4, the apparatus includes animage obtaining portion 100, a coordinate value generating portion 102,a sensing portion 104, a movement amount generating portion 106, aposture updating portion 108, a control portion 110, and a positionchecking portion 112. An apparatus plays the role of performing themethod shown in FIG. 1.

To execute Step 10 of FIG. 1, the control portion 110 generates amovement control signal and the generated movement control signal isoutput through an output port OUT1. Thus, the mobile device starts tomove in response to the movement control signal.

The control portion 110 and the posture updating portion 108 performStep 12. That is, the control portion 110 generates a postureinitialization signal and the generated posture initialization signal isoutput through the posture updating portion 108. The posture updatingportion 108 initializes the posture, that is, information on theposition and pose of the mobile device, in response to the postureinitialization signal output from the control portion 110.

To execute Step 14, in response to an obtainment control signalgenerated by and output from the control portion 110, the imageobtaining portion 100 obtains images at two different points on a pathalong which the mobile device moves and outputs the obtained images tothe coordinate value generating portion 102. For example, when theinitialization of the mobile device is recognized to be completedthrough the obtainment control signal, the image obtaining portion 100obtains the two images. To this end, the image obtaining portion 100 maybe at least one camera (not shown) attached to the mobile device. Whilemoving with the mobile device, the camera obtains two images from twodifferent points.

To execute Step 16, the coordinate value generating portion 102 searchesfor matching points with respect to the obtained images, generates animage coordinate value of each matching point, and outputs the generatedimage coordinate value to the movement amount generating portion 106.The matching points found by the coordinate value generating portion 102may correspond to the minimum required matching points.

The sensing portion 104 and the movement amount generating portion 106perform Step 18. The sensing portion 104 senses linear movement of themobile device and outputs the sensed result to the movement amountgenerating portion 106. For example, when the mobile device adopts awheel mechanism moving using wheels, the sensing portion 104 may be awheel encoder (not shown). However, when the mobile device is a flyingobject, the sensing portion 104 may be a linear accelerometer (notshown). The movement amount generating portion 106 generates a linearmovement amount from the result sensed by the sensing portion. 104 andoutputs the generated linear movement amount to the posture updatingportion 108.

To execute Step 20, the movement amount generating portion 106 generatesa rotational movement amount from the generated linear movement amountand the image coordinate values output from the coordinate valuegenerating portion 102, and outputs the generated rotational movementamount to the posture updating portion 108.

To execute Step 22, the posture updating portion 108 updates the postureof the mobile device from the rotational movement amount and the linearmovement amount generated by the movement amount generating portion 106,and outputs the updated result through an output port OUT2.

To execute Step 24, the position checking portion 112, in response to anupdate completion signal, checks whether the mobile device arrives atthe target position and outputs the checked result to the imageobtaining portion 100. That is, when the completion of the update of theposture is recognized through the update completion signal output fromthe posture updating portion 108, the position checking portion 112checks whether the mobile device arrives at the target position.

The image obtaining portion 100 obtains an image in response to theresult checked by the position checking portion 112 (Step 14). That is,when it is recognized from the result of checking output from theposition checking portion 112 that the mobile device does not arrive atthe target position, two images at different two points on the pathalong which the mobile device moves are obtained.

Like the method shown in FIG. 1 which selectively adopts Steps 10, 12,22, and 24, the apparatus shown in FIG. 3 may selectively adopt theposture updating portion 108 and the position checking portion 112 andrestrict the operation of the control portion 110 to selectively executeSteps 10 and 12.

FIG. 5 is a block diagram illustrating an apparatus for using arotational movement amount of a mobile device according to anotherembodiment of the present invention. The apparatus shown in FIG. 5includes an image obtaining portion 150, a coordinate value generatingportion 152, a sensing portion 154, a movement amount generating portion156, a posture updating portion 158, a correction checking portion 160,a control portion 162, and a position checking portion 164.

The apparatus shown in FIG. 5 performs the method shown in FIG. 3. Sincethe coordinate value generating portion 152, the sensing portion 154,the movement amount generating portion 156, and the posture updatingportion 158 shown in FIG. 5 have the same functions as those of thecoordinate value generating portion 102, the sensing portion 104, themovement amount generating portion 106, and the posture updating portion108 shown in FIG. 4, detailed descriptions thereof will be omitted.

To execute Step 50, the control portion 162 generates a movement controlsignal and outputs the generated movement control signal through anoutput port OUT3. Thus, the mobile device starts to move in response tothe movement control signal.

The control portion 162 and the posture updating portion 158 executeStep 52. First, the control portion 162 generates a postureinitialization signal and outputs the generated posture initializationsignal to the posture updating portion 158. The posture updating portion158 initializes posture in response to the posture initializationsignal.

To execute Step 54, in response to a correction control signal generatedby the control portion 162, the correction checking portion 160 checkswhether to correct the posture of the mobile device and outputs theresult of checking to the image obtaining portion 150 and the controlportion 162. For example, when the completion of the initialization ofthe mobile device is recognized through the correction control signal,the correction checking portion 160 starts to check whether to correctthe posture of the mobile device.

To execute Step 56, the image obtaining portion 150, in response to theresult of checking an output from the correction checking portion 160,obtains images and outputs the obtained images to the coordinate valuegenerating portion 152. That is, when it is recognized from the resultof checking output from the correction checking portion 160 that theposture needs to be corrected, the image obtaining portion 150 obtainstwo images.

To execute Step 66, the position checking portion 164, in response to anupdate completion signal output from the posture updating portion 158,checks whether the mobile device arrives at the target position andoutputs the result of checking to the correction checking portion 160.That is, if the it is recognized from the update completion signal thatthe updating of the posture is completed, the position checking portion164 checks whether the mobile device arrives the target position.

To execute Step 54, the correction checking portion 160, in response tothe result of checking by the position checking portion 164, checkswhether the posture is corrected. That is, if it is recognized from theresult of checking by the position checking portion 164 that the mobiledevice does not arrived at the target position, the correction checkingportion 160 checks whether to correct the posture.

To execute Step 68, the control portion 162 generates a movement controlsignal in response to the result of checking by the correction checkingportion 160. That is, if it is recognized from the result of checking bythe correction checking portion 160 that the posture does not need to becorrected, the control portion 162 generates the movement control signalto move the mobile device and outputs the movement control signalthrough the output port OUT3. After executing Step 68, the controlportion 162 controls the position checking portion 164 so that theposition checking portion 164 executes Step 66.

A computer-readable recording medium for storing at least one computerprogram to control the above-described apparatus for using a rotationalmovement amount is described below.

The computer program stored in a computer capable of reading a recordingmedium according to the present invention executes the steps ofobtaining images at two different points on a path along which a mobiledevice moves, searching for matching points with respect to the obtainedimages and generating an image coordinate value of each matching point,generating a linear movement amount by sensing linear movement of themobile device, and generating a rotational movement amount using theimage coordinate values and the linear movement amount.

The computer program may further execute a step of updating posture ofthe mobile device using the linear movement amount and the rotationalmovement amount. Moreover, the computer program stored on the recordingmedium may further execute a step of determining whether to correct theposture of the mobile device. In this case, if it is determined that theposture of the mobile device needs to be corrected, the process goes tothe step of obtaining the images. Otherwise, the mobile device is moved.

The method and apparatus for using a rotational movement amount of amobile device and a computer-readable recording medium for storing acomputer program according to the present invention can be applied to anself-running technology that is a core technology of a mobile robot,development of an environment recognition technology, an unmanned flyingsystem, or an extremely cold or hot area expedition equipment.

As described above, the method and apparatus for using a rotationalmovement amount of a mobile device and a computer-readable recordingmedium for storing a computer program according to the presentinvention, through a chief structure consisting of the sensor 104 or 154sensing the linear movement amount of the mobile device and the imageobtaining portion 100 or 150 obtaining images, for example, a camera,estimate the rotational movement amount of the mobile device to locallyrecognize the posture. Unlike the conventional method in which an erroris accumulated according to time by repeated integrations, an error inestimating the rotation movement amount is within a bounded range. Inparticular, when the mobile device moves on a plane, in obtaining asolution of the Epipolar Geometry, the number of the matching points tobe searched for can be reduced by ¼ compared to the case of using onlyimages of a camera so that a calculation time is reduced. Also, thepresent invention has an effect of high applicability in development ofa self-running system in real time of a mobile device, for example, arobot.

1. A method of determining a rotational movement amount of a mobiledevice comprising: generating a posture initialization signal of themobile device; determining whether a posture of the mobile device isrequired by using a correction checking device; obtaining images at twodifferent points on a path along which the mobile device moves by usingan image obtaining device in response to the correction checking unitwhen a correction of the posture of the mobile device is required;searching for matching points with respect to the obtained images andobtaining an image coordinate value of each of the matching points;sensing linear movement of the mobile device and obtaining a linearmovement amount from a sensing device; and obtaining the rotationalmovement amount by calculating the image coordinate values together withthe linear movement amount; and updating posture of the mobile deviceusing the linear movement amount and the rotational movement amount byusing a posture updating device; determining whether the mobile devicearrived at a target position by using a position checking device andoutputting a result to the correction checking device; and generating amovement control signal by using a control unit upon determining thecorrection of the posture of the mobile device is not required, whereinthe calculating of the image coordinate values together with the linearmovement amount includes multiplying the image coordinate values with across product calculation of the linear movement amount.
 2. The methodof claim 1, wherein the mobile device moves from a time at which theimage coordinate value is obtained until the posture is updated.
 3. Themethod of claim 1, wherein the mobile device stops from a time at whichthe image coordinate value is obtained until the posture is updated. 4.The method of claim 1, further comprising: starting movement of themobile device.
 5. The method of claim 1, further comprising moving themobile device and proceeding to determining whether the mobile devicearrives at the target position if it is determined that the posture ofthe mobile device does not need to be corrected.
 6. The method of claim1, wherein the number of matching points is eight and the number ofminimum required matching points is eight.
 7. The method of claim 6,wherein the number of matching points is set to be an integral multipleof the number of minimum required matching points.
 8. The method ofclaim 1, wherein the number of matching points is two and the number ofa minimum required matching points is two.
 9. The method of claim 1,wherein path formed by the two points is a straight line.
 10. The methodof claim 1, wherein the rotational movement amount of the mobile deviceis obtained using a conversion matrix (R_(1,2)) that is obtained asfollows, $\begin{matrix}{{{\left\lbrack {\frac{\overset{\_}{u_{i,1}}}{f_{sx}}a_{i}\frac{\overset{\_}{v_{i,1}}}{f_{sy}}} \right\rbrack \cdot R_{2,1} \cdot \left\lbrack {\frac{u_{i,2}}{f_{sx}}1\frac{v_{i,2}}{f_{sy}}} \right\rbrack^{T}} = 0},} \\{{{wherein}\mspace{14mu}\left\lbrack {\frac{\overset{\_}{u_{i,1}}}{f_{sx}}a_{i}\frac{\overset{\_}{v_{i,1}}}{f_{sy}}} \right\rbrack} = {\left\lbrack {\frac{u_{i,1}}{f_{sx}}1\frac{v_{i,1}}{f_{sy}}} \right\rbrack{s\left( {{}_{}^{}{}_{1,2}^{}} \right)}}}\end{matrix}$ and i=1, . . . , l,l denotes the number of the minimumrequired matching points, and ¹t_(1,2) denotes a linear movement amountexpressed as Coord. 1, u_(i,2), u_(i,1), v_(i,1), and v_(i,2) denote theimage coordinate values, s( ) denotes a skew-symmetric matrix defined toexpress the cross product calculation of a vector, R_(2,1) is an inversematrix of R_(1,2),${f_{sx} = \frac{f_{1}}{s_{x}}},{f_{sy} = \frac{f_{2}}{s_{y}}},$ f₁ andf₂ denote focal distances from the two points, s_(x) and s_(y) denotescale factors, T denotes a transposed matrix, and a₁ denotes a constant.11. An apparatus for determining a rotational movement amount of amobile device comprising: a correction checking portion which determineswhether a correction of a posture of the mobile device is required; animage obtaining portion which obtains images from two different pointson a path along which the mobile device moves, upon determining that theposture of the mobile device is required; a coordinate value generatingportion which searches for matching points with respect to the obtainedimages and generates an image coordinate value for each of the matchingpoints; a sensing portion which senses linear movement of the mobiledevice; a movement amount generating portion which generates a linearmovement amount from a result of sensing and generates the rotationalmovement amount by calculating the image coordinate values together withthe linear movement amount; a posture updating portion which updatesposture of the mobile device from the linear movement amount and therotational movement amount generated by the movement amount generatingportion and outputs an updated result; a position checking portion whichdetermines whether the mobile device arrived at a target position andoutputting a result to the correction checking portion; and a controlportion generating a movement control signal upon determining that thecorrection of the posture of the mobile device is not required, andgenerating a posture initialization signal, wherein the calculating ofthe image coordinate values together with the linear movement amountincludes multiplying the image coordinate values with a cross productcalculation of the linear movement amount.
 12. The apparatus of claim11, wherein the image obtaining portion obtains the images in responseto a result of determining of the correction checking portion.
 13. Theapparatus of claim 12, wherein the control portion generates acorrection control signal; and the position checking portion checkswhether the mobile device arrives at a target position in response to anupdate completion signal that indicates completion of update of theposture and is received from the posture updating portion, wherein themobile device moves in response to the movement control signal, theposture updating portion initializes the posture in response to theposture initialization signal, and the correction checking portionchecks whether to correct the posture in response to the correctioncontrol signal or a result of checking checked by the position checkingportion.
 14. The apparatus of claim 13, wherein the control portiongenerates the movement control signal in response to a result ofchecking by the correction checking portion and controls a checkingoperation of the position checking portion.
 15. The apparatus of claim11, wherein the control portion which generates an obtainment controlsignal; and the position checking portion which checks whether themobile device arrives at a target position in response to an updatecompletion signal that indicates completion of update and is receivedfrom the posture updating portion, wherein the mobile device moves inresponse to the movement control signal, the posture updating portioninitializes the posture in response to the posture initializationsignal, and the image obtaining portion obtains the image in response tothe obtainment control signal and a result of checking by the positionchecking portion.
 16. A non-transitory computer-readable storage mediumstoring at least one computer program to control at least one processingdevice to implement a method of determining a rotational movement amountof a mobile device, the method comprising: determining whether a postureof the mobile device is required; obtaining images at two differentpoints on a path along which the mobile device moves, upon determiningthat the posture of the mobile device is required; searching formatching points with respect to the obtained images and generating animage coordinate value of each of the matching points; sensing linearmovement of the mobile device and generating a linear movement amountfrom the sensing; generating the rotational movement amount bycalculating the image coordinate values together with the linearmovement amount; updating posture of the mobile device using the linearmovement amount and the rotational movement amount, determining whetherthe mobile device arrived at a target position and outputting a result;and generating a movement control signal upon determining that thecorrection of the posture of the mobile device is not required, whereinthe calculating of the image coordinate values together with the linearmovement amount includes multiplying the image coordinate values with across product calculation of the linear movement amount.